{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras\n",
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"http://nbviewer.ipython.org/github/ShowMeAI-Hub/awesome-AI-cheatsheets/blob/main/Keras/Keras-cheatsheet-code.ipynb\"><img src=\"https://raw.githubusercontent.com/jupyter/design/master/logos/Badges/nbviewer_badge.svg\" />在nbviewer上查看notebook</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/ShowMeAI-Hub/awesome-AI-cheatsheets/blob/main/Keras/Keras-cheatsheet-code.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" />在Google Colab运行</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/ShowMeAI-Hub/awesome-AI-cheatsheets/tree/main/Keras/Keras-cheatsheet-code.ipynb\"><img src=\"https://badgen.net/badge/open/github/color=cyan?icon=github\" />在Github上查看源代码</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/ShowMeAI-Hub/awesome-AI-cheatsheets/Keras/Keras速查表.pdf\"><img src=\"https://badgen.net/badge/download/pdf/color=white?icon=github\"/>下载速查表</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 说明\n",
    "**notebook by [韩信子](https://github.com/HanXinzi-AI)@[ShowMeAI](https://github.com/ShowMeAI-Hub)**\n",
    "\n",
    "更多AI速查表资料请查看[速查表大全](https://github.com/ShowMeAI-Hub/awesome-AI-cheatsheets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keras是强大、易用的深度学习库，基于Theano和TensorFlow提供了高阶神经网络API，用于开发和评估深度学习模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 典型示例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1000/1000 [==============================] - 0s 119us/step - loss: 0.6994 - accuracy: 0.5130\n",
      "Epoch 2/10\n",
      "1000/1000 [==============================] - 0s 28us/step - loss: 0.6916 - accuracy: 0.5400\n",
      "Epoch 3/10\n",
      "1000/1000 [==============================] - 0s 29us/step - loss: 0.6846 - accuracy: 0.5370\n",
      "Epoch 4/10\n",
      "1000/1000 [==============================] - 0s 25us/step - loss: 0.6782 - accuracy: 0.5560\n",
      "Epoch 5/10\n",
      "1000/1000 [==============================] - 0s 23us/step - loss: 0.6759 - accuracy: 0.5710\n",
      "Epoch 6/10\n",
      "1000/1000 [==============================] - 0s 25us/step - loss: 0.6695 - accuracy: 0.5830\n",
      "Epoch 7/10\n",
      "1000/1000 [==============================] - 0s 24us/step - loss: 0.6690 - accuracy: 0.5890\n",
      "Epoch 8/10\n",
      "1000/1000 [==============================] - 0s 25us/step - loss: 0.6603 - accuracy: 0.6150\n",
      "Epoch 9/10\n",
      "1000/1000 [==============================] - 0s 23us/step - loss: 0.6564 - accuracy: 0.6090\n",
      "Epoch 10/10\n",
      "1000/1000 [==============================] - 0s 25us/step - loss: 0.6571 - accuracy: 0.6200\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from keras.models import Sequential     #顺序模型\n",
    "from keras.layers import Dense    #全连接层\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "data = np.random.random((1000,100))   #数据\n",
    "labels = np.random.randint(2,size=(1000,1))  #标签\n",
    "\n",
    "model = Sequential()   #初始化顺序模型\n",
    "model.add(Dense(32, activation='relu', input_dim=100))   #添加全连接层\n",
    "model.add(Dense(1, activation='sigmoid'))   #添加二分类全连接层\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])   #编译模型\n",
    "model.fit(data,labels,epochs=10,batch_size=32)  #拟合数据\n",
    "predictions = model.predict(data)   #预估数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据加载"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据要存为 NumPy 数组或数组列表，使用 sklearn.cross_validation 的 train_test_split 模块进行分割将数据分割为训练集与测试集。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Keras 数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import boston_housing, mnist, cifar10, imdb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train,y_train),(x_test,y_test) = mnist.load_data()          #手写数字数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train2,y_train2),(x_test2,y_test2) = boston_housing.load_data()     #波士顿房价数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train3,y_train3),(x_test3,y_test3) = cifar10.load_data()   #cifar图像分类数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train4,y_train4),(x_test4,y_test4) = imdb.load_data(num_words=20000)     #imdb评论数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_classes = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 其它"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"https://github.com/npradaschnor/Pima-Indians-Diabetes-Dataset/raw/master/diabetes.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pregnancies</th>\n",
       "      <th>Glucose</th>\n",
       "      <th>BloodPressure</th>\n",
       "      <th>SkinThickness</th>\n",
       "      <th>Insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>DiabetesPedigreeFunction</th>\n",
       "      <th>Age</th>\n",
       "      <th>Outcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6</td>\n",
       "      <td>148</td>\n",
       "      <td>72</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>33.6</td>\n",
       "      <td>0.627</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>85</td>\n",
       "      <td>66</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>26.6</td>\n",
       "      <td>0.351</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>183</td>\n",
       "      <td>64</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23.3</td>\n",
       "      <td>0.672</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>89</td>\n",
       "      <td>66</td>\n",
       "      <td>23</td>\n",
       "      <td>94</td>\n",
       "      <td>28.1</td>\n",
       "      <td>0.167</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>137</td>\n",
       "      <td>40</td>\n",
       "      <td>35</td>\n",
       "      <td>168</td>\n",
       "      <td>43.1</td>\n",
       "      <td>2.288</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pregnancies  Glucose  BloodPressure  SkinThickness  Insulin   BMI  \\\n",
       "0            6      148             72             35        0  33.6   \n",
       "1            1       85             66             29        0  26.6   \n",
       "2            8      183             64              0        0  23.3   \n",
       "3            1       89             66             23       94  28.1   \n",
       "4            0      137             40             35      168  43.1   \n",
       "\n",
       "   DiabetesPedigreeFunction  Age  Outcome  \n",
       "0                     0.627   50        1  \n",
       "1                     0.351   31        0  \n",
       "2                     0.672   32        1  \n",
       "3                     0.167   21        0  \n",
       "4                     2.288   33        1  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = data.loc[:,'Pregnancies':'Age']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = data.loc[:,'Outcome']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据预处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 序列填充"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.preprocessing import sequence\n",
    "\n",
    "x_train4 = sequence.pad_sequences(x_train4, maxlen=80)   #填充为固定长度80的序列\n",
    "x_test4 = sequence.pad_sequences(x_test4, maxlen=80) #填充为固定长度80的序列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练与测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train5,X_test5,y_train5,y_test5 = train_test_split(X, y, test_size=0.33, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 独热编码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils import to_categorical\n",
    "\n",
    "Y_train = to_categorical(y_train, num_classes)   #类别标签独热编码转换\n",
    "Y_test = to_categorical(y_test, num_classes)  #类别标签独热编码转换\n",
    "\n",
    "Y_train3 = to_categorical(y_train3, num_classes)  #类别标签独热编码转换\n",
    "Y_test3 = to_categorical(y_test3, num_classes) #类别标签独热编码转换"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 标准化/归一化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler().fit(x_train2)\n",
    "standardized_X = scaler.transform(x_train2)\n",
    "standardized_X_test = scaler.transform(x_test2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型架构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 顺序模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "\n",
    "model = Sequential()\n",
    "model2 = Sequential()\n",
    "model3 = Sequential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多层感知器（MLP）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**二进制分类**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Dense\n",
    "\n",
    "model.add(Dense(12, input_dim=8, kernel_initializer='uniform', activation='relu'))  #添加12个神经元的全连接层\n",
    "model.add(Dense(8,kernel_initializer='uniform',activation='relu')) #添加8个神经元的全连接层\n",
    "model.add(Dense(1,kernel_initializer='uniform',activation='sigmoid')) #二分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**多级分类**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Dropout\n",
    "\n",
    "model.add(Dense(512,activation='relu',input_shape=(784,))) #添加512个神经元的全连接层\n",
    "model.add(Dropout(0.2))  #添加随机失活层\n",
    "model.add(Dense(512,activation='relu'))  #添加512个神经元的全连接层\n",
    "model.add(Dropout(0.2))  #添加随机失活层\n",
    "model.add(Dense(10,activation='softmax'))  #10分类的全连接层"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**回归**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(64,activation='relu',input_dim=x_train2.shape[1]))  #添加64个神经元的全连接层\n",
    "model.add(Dense(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积神经网络（CNN）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Activation,Conv2D,MaxPooling2D,Flatten\n",
    "\n",
    "model2.add(Conv2D(32,(3,3),padding='same',input_shape=(28,28,1,)))  #2D卷积层\n",
    "model2.add(Activation('relu'))  #ReLU激活函数\n",
    "model2.add(Conv2D(32,(3,3)))  #2D卷积层\n",
    "model2.add(Activation('relu'))  #ReLU激活函数\n",
    "model2.add(MaxPooling2D(pool_size=(2,2)))  #2D池化层\n",
    "model2.add(Dropout(0.25))  #添加随机失活层\n",
    "model2.add(Conv2D(64,(3,3), padding='same'))  #2D卷积层\n",
    "model2.add(Activation('relu'))  #ReLU激活函数\n",
    "model2.add(Conv2D(64,(3, 3)))  #2D卷积层\n",
    "model2.add(Activation('relu'))  #ReLU激活函数\n",
    "model2.add(MaxPooling2D(pool_size=(2,2)))  #2D池化层\n",
    "model2.add(Dropout(0.25))  #添加随机失活层\n",
    "model2.add(Flatten())  #展平成vector\n",
    "model2.add(Dense(512)) #全连接层\n",
    "model2.add(Activation('relu')) #ReLU激活函数\n",
    "model2.add(Dropout(0.5)) #添加随机失活层\n",
    "model2.add(Dense(num_classes))  #类别数个神经元的全连接层\n",
    "model2.add(Activation('softmax'))  #softmax多分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 递归神经网络（RNN）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Embedding,LSTM\n",
    "\n",
    "model3.add(Embedding(20000,128))  #嵌入层\n",
    "model3.add(LSTM(128,dropout=0.2,recurrent_dropout=0.2))  #LSTM层\n",
    "model3.add(Dense(1,activation='sigmoid')) #二分类全连接"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 审视模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**获取模型信息**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(None, 1)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.output_shape   #模型输出形状"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_3 (Dense)              (None, 12)                108       \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 8)                 104       \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 1)                 9         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 512)               1024      \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 512)               262656    \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 10)                5130      \n",
      "_________________________________________________________________\n",
      "dense_9 (Dense)              (None, 64)                704       \n",
      "_________________________________________________________________\n",
      "dense_10 (Dense)             (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 269,800\n",
      "Trainable params: 269,800\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()   #模型摘要展示"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'sequential_2',\n",
       " 'layers': [{'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_3',\n",
       "    'trainable': True,\n",
       "    'batch_input_shape': (None, 8),\n",
       "    'dtype': 'float32',\n",
       "    'units': 12,\n",
       "    'activation': 'relu',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'RandomUniform',\n",
       "     'config': {'minval': -0.05, 'maxval': 0.05, 'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}},\n",
       "  {'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_4',\n",
       "    'trainable': True,\n",
       "    'dtype': 'float32',\n",
       "    'units': 8,\n",
       "    'activation': 'relu',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'RandomUniform',\n",
       "     'config': {'minval': -0.05, 'maxval': 0.05, 'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}},\n",
       "  {'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_5',\n",
       "    'trainable': True,\n",
       "    'dtype': 'float32',\n",
       "    'units': 1,\n",
       "    'activation': 'sigmoid',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'RandomUniform',\n",
       "     'config': {'minval': -0.05, 'maxval': 0.05, 'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}},\n",
       "  {'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_6',\n",
       "    'trainable': True,\n",
       "    'batch_input_shape': (None, 784),\n",
       "    'dtype': 'float32',\n",
       "    'units': 512,\n",
       "    'activation': 'relu',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'VarianceScaling',\n",
       "     'config': {'scale': 1.0,\n",
       "      'mode': 'fan_avg',\n",
       "      'distribution': 'uniform',\n",
       "      'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}},\n",
       "  {'class_name': 'Dropout',\n",
       "   'config': {'name': 'dropout_1',\n",
       "    'trainable': True,\n",
       "    'dtype': 'float32',\n",
       "    'rate': 0.2,\n",
       "    'noise_shape': None,\n",
       "    'seed': None}},\n",
       "  {'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_7',\n",
       "    'trainable': True,\n",
       "    'dtype': 'float32',\n",
       "    'units': 512,\n",
       "    'activation': 'relu',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'VarianceScaling',\n",
       "     'config': {'scale': 1.0,\n",
       "      'mode': 'fan_avg',\n",
       "      'distribution': 'uniform',\n",
       "      'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}},\n",
       "  {'class_name': 'Dropout',\n",
       "   'config': {'name': 'dropout_2',\n",
       "    'trainable': True,\n",
       "    'dtype': 'float32',\n",
       "    'rate': 0.2,\n",
       "    'noise_shape': None,\n",
       "    'seed': None}},\n",
       "  {'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_8',\n",
       "    'trainable': True,\n",
       "    'dtype': 'float32',\n",
       "    'units': 10,\n",
       "    'activation': 'softmax',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'VarianceScaling',\n",
       "     'config': {'scale': 1.0,\n",
       "      'mode': 'fan_avg',\n",
       "      'distribution': 'uniform',\n",
       "      'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}},\n",
       "  {'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_9',\n",
       "    'trainable': True,\n",
       "    'batch_input_shape': (None, 13),\n",
       "    'dtype': 'float32',\n",
       "    'units': 64,\n",
       "    'activation': 'relu',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'VarianceScaling',\n",
       "     'config': {'scale': 1.0,\n",
       "      'mode': 'fan_avg',\n",
       "      'distribution': 'uniform',\n",
       "      'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}},\n",
       "  {'class_name': 'Dense',\n",
       "   'config': {'name': 'dense_10',\n",
       "    'trainable': True,\n",
       "    'dtype': 'float32',\n",
       "    'units': 1,\n",
       "    'activation': 'linear',\n",
       "    'use_bias': True,\n",
       "    'kernel_initializer': {'class_name': 'VarianceScaling',\n",
       "     'config': {'scale': 1.0,\n",
       "      'mode': 'fan_avg',\n",
       "      'distribution': 'uniform',\n",
       "      'seed': None}},\n",
       "    'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n",
       "    'kernel_regularizer': None,\n",
       "    'bias_regularizer': None,\n",
       "    'activity_regularizer': None,\n",
       "    'kernel_constraint': None,\n",
       "    'bias_constraint': None}}]}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.get_config()   #模型配置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([[ 0.02261684,  0.00681395, -0.00799289,  0.04708118,  0.01690466,\n",
       "         -0.01600778,  0.03737886, -0.01720665, -0.04903277, -0.04005467,\n",
       "          0.00557697,  0.04961372],\n",
       "        [-0.01949078,  0.0424651 , -0.04685105, -0.03737346,  0.03238528,\n",
       "         -0.00384414, -0.01554109,  0.03094954,  0.04226147,  0.00926924,\n",
       "         -0.00980644,  0.00260032],\n",
       "        [ 0.04117329,  0.04754898, -0.03944343, -0.00947599, -0.01077851,\n",
       "         -0.04226322,  0.03801937, -0.03478085,  0.01588634, -0.03343659,\n",
       "         -0.02843529,  0.01413815],\n",
       "        [ 0.02689786, -0.01126499, -0.02619879,  0.02608598, -0.0016798 ,\n",
       "         -0.02144979, -0.02570363, -0.03729799,  0.02673054, -0.0212526 ,\n",
       "          0.03346172,  0.04175923],\n",
       "        [ 0.01401086, -0.03008029, -0.01921008, -0.01720281, -0.02832749,\n",
       "          0.00544517,  0.03468115,  0.03147023,  0.0082023 , -0.03249103,\n",
       "         -0.01127851,  0.01755785],\n",
       "        [ 0.04308366,  0.02193056,  0.01572079,  0.02118572,  0.018023  ,\n",
       "         -0.02658724, -0.0413038 ,  0.00767887,  0.01725108, -0.00920826,\n",
       "         -0.03181674, -0.01079444],\n",
       "        [-0.01773909,  0.03443897,  0.0219262 , -0.0020633 , -0.00601583,\n",
       "         -0.04137429,  0.00101211, -0.03070215,  0.03441885, -0.0113625 ,\n",
       "         -0.03569374, -0.02303358],\n",
       "        [-0.00300644, -0.0339596 , -0.02531221, -0.03565655, -0.03900003,\n",
       "         -0.01491892, -0.03967806,  0.00148762, -0.02731364, -0.03145832,\n",
       "          0.00706154,  0.03701838]], dtype=float32),\n",
       " array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32),\n",
       " array([[-0.04123209, -0.00957907, -0.02940718, -0.03089998, -0.01886008,\n",
       "         -0.01766922, -0.02467432, -0.03077017],\n",
       "        [ 0.03001746,  0.01603216, -0.01450461,  0.04420792,  0.02918993,\n",
       "          0.02404395,  0.02953383,  0.03853163],\n",
       "        [-0.02035526, -0.00861864, -0.01100037, -0.03751143,  0.03190893,\n",
       "         -0.01816846, -0.01729505, -0.02803063],\n",
       "        [ 0.01088298, -0.02108314,  0.03909013, -0.0143445 , -0.04564939,\n",
       "         -0.01608121,  0.01125135,  0.0408368 ],\n",
       "        [-0.00532136,  0.03909772,  0.00397488, -0.03817874, -0.00087072,\n",
       "         -0.00654322,  0.01292379,  0.03107751],\n",
       "        [-0.02122883, -0.02971569, -0.01778457,  0.02034484,  0.00506121,\n",
       "         -0.00998557, -0.02001989, -0.00778143],\n",
       "        [-0.03013797, -0.03195975, -0.03278799,  0.02974215,  0.02008354,\n",
       "          0.02891502, -0.00158061, -0.00963949],\n",
       "        [-0.03329402,  0.02038309,  0.04944451, -0.03416421,  0.04157503,\n",
       "         -0.0425902 ,  0.01490916, -0.03099825],\n",
       "        [ 0.02645234,  0.04343052, -0.03319963,  0.01547307, -0.04041018,\n",
       "         -0.02669707, -0.02180594, -0.04225973],\n",
       "        [ 0.00015552,  0.00721375,  0.03774362,  0.04289123, -0.02686647,\n",
       "         -0.00518777, -0.01966482, -0.02225661],\n",
       "        [-0.00966341, -0.02191113, -0.04064469, -0.04428265, -0.0443262 ,\n",
       "         -0.00364801, -0.03309365, -0.03383507],\n",
       "        [-0.01261333,  0.01553581, -0.00936819, -0.01778264, -0.04140538,\n",
       "         -0.04207589, -0.04317564,  0.02156887]], dtype=float32),\n",
       " array([0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32),\n",
       " array([[-2.8488709e-02],\n",
       "        [ 1.7656039e-02],\n",
       "        [ 1.9349158e-05],\n",
       "        [ 2.5446836e-02],\n",
       "        [-5.9114806e-03],\n",
       "        [-3.8863480e-02],\n",
       "        [ 2.6876900e-02],\n",
       "        [-2.3385871e-02]], dtype=float32),\n",
       " array([0.], dtype=float32),\n",
       " array([[ 0.05785965,  0.08209214,  0.10601235,  0.06550107, -0.01880015,\n",
       "         -0.01903778,  0.03122968,  0.04040789,  0.03033198,  0.05821063,\n",
       "         -0.07772297, -0.00099706,  0.02766392,  0.06605934, -0.10720102,\n",
       "         -0.10498199,  0.02406058,  0.05588794,  0.01017883, -0.08347423,\n",
       "         -0.03346261,  0.07940987, -0.09951992,  0.04899467, -0.00198431,\n",
       "          0.02495136, -0.09834877, -0.03395076, -0.09676847,  0.0082498 ,\n",
       "         -0.06381678, -0.07946473, -0.0008572 ,  0.06792559, -0.00639603,\n",
       "          0.04091325,  0.04814035,  0.02481139, -0.01651119, -0.02947397,\n",
       "         -0.04867342, -0.07164209,  0.06067214,  0.08209332,  0.0101515 ,\n",
       "          0.08483834, -0.00344699, -0.08884755,  0.02047109, -0.04267187,\n",
       "         -0.00297833, -0.03577423,  0.10068048,  0.02067921, -0.02536087,\n",
       "         -0.02365255,  0.01808754, -0.10101253, -0.03602981, -0.03925448,\n",
       "          0.08797792, -0.03061433,  0.0038486 , -0.0527504 , -0.07864802,\n",
       "         -0.01834518,  0.02627771, -0.0672165 , -0.09750781, -0.02090264,\n",
       "         -0.03918131,  0.02259497, -0.03792509,  0.01343212,  0.06831407,\n",
       "          0.00134945,  0.02979413,  0.0879618 , -0.10428607,  0.0160439 ,\n",
       "         -0.09821895, -0.02703218, -0.09958077,  0.05484927, -0.09579995,\n",
       "          0.10685786, -0.09210716,  0.08949517, -0.08386668, -0.03116413,\n",
       "         -0.07189629,  0.00849021, -0.03303291,  0.06701083,  0.08020944,\n",
       "          0.06391648,  0.00322787, -0.04365343, -0.00641848,  0.09777435,\n",
       "         -0.02511233, -0.08901876,  0.04911632,  0.09801076, -0.10161204,\n",
       "         -0.06670651,  0.08991041,  0.10657778,  0.03563529,  0.09353023,\n",
       "         -0.02180035, -0.06726956,  0.0075045 ,  0.03217038,  0.08676232,\n",
       "         -0.09719569, -0.03562027, -0.04747143,  0.0090307 ,  0.04026624,\n",
       "         -0.01112156, -0.01116712, -0.01796072, -0.09586158,  0.01313256,\n",
       "         -0.02656381, -0.02332687, -0.03026511, -0.00218164,  0.00503523,\n",
       "          0.10243054, -0.01301008, -0.05219515,  0.06384455, -0.02035464,\n",
       "          0.04670315, -0.04619133,  0.00715742,  0.09925029,  0.06690147,\n",
       "          0.0606707 , -0.04461042,  0.08287377,  0.01430771,  0.01898033,\n",
       "         -0.0276941 , -0.09921402, -0.0821984 ,  0.10260757, -0.05189018,\n",
       "         -0.03712244,  0.04424884,  0.08694568, -0.10369171, -0.01207963,\n",
       "          0.01098748, -0.04459649, -0.00675386, -0.0180825 , -0.02447332,\n",
       "          0.06120021,  0.03483864,  0.08819913,  0.0149467 , -0.08890025,\n",
       "         -0.09277967, -0.04094921,  0.03891281,  0.07577848, -0.05019882,\n",
       "          0.00203261, -0.00065938,  0.058923  ,  0.00563082,  0.00243614,\n",
       "         -0.00967578, -0.00229048, -0.10366198,  0.04770338,  0.0763858 ,\n",
       "         -0.05181888,  0.08193936, -0.03660441,  0.04110704,  0.06705987,\n",
       "         -0.01346833,  0.07922644, -0.05772318, -0.05920078,  0.04055125,\n",
       "         -0.061841  ,  0.0997385 ,  0.10344715,  0.08626182, -0.00565075,\n",
       "         -0.03753399,  0.01685264, -0.10567151, -0.01386835,  0.04021106,\n",
       "         -0.00093793,  0.00217551, -0.07714004, -0.05459767, -0.08189894,\n",
       "         -0.1060005 ,  0.00235976, -0.0460184 ,  0.05128133,  0.09301888,\n",
       "          0.00012606,  0.02101142, -0.07718143, -0.05241283,  0.06139425,\n",
       "          0.10674167,  0.09238135,  0.00130296, -0.05025849, -0.04196888,\n",
       "          0.08026046, -0.07582729,  0.07540684, -0.04179069, -0.07398945,\n",
       "         -0.01267151,  0.02644175,  0.09067795, -0.09080613, -0.04562782,\n",
       "          0.01044921, -0.00224905,  0.01864392,  0.08991087, -0.08808604,\n",
       "         -0.06295612, -0.0066435 , -0.09364516,  0.01444445, -0.00299431,\n",
       "          0.01106501, -0.03134844, -0.00945501,  0.01884053,  0.02719151,\n",
       "          0.10063948, -0.05858763, -0.02598315,  0.03000427, -0.0144075 ,\n",
       "         -0.03768952,  0.05675221,  0.05248626,  0.01301555, -0.08849823,\n",
       "          0.10501713,  0.09429561,  0.00255639,  0.08034805,  0.10042336,\n",
       "          0.01519483,  0.04711019, -0.03296827, -0.00516675, -0.0142654 ,\n",
       "          0.03575056,  0.0602997 ,  0.04489722,  0.01860412, -0.06134627,\n",
       "         -0.01506111, -0.10084429, -0.06699589,  0.00655574,  0.08852073,\n",
       "         -0.03664739,  0.08849735,  0.00789343, -0.10364713, -0.09182011,\n",
       "         -0.10727654,  0.05658837,  0.00722038, -0.05804272, -0.02680737,\n",
       "          0.06062438, -0.0447898 , -0.02854183, -0.02186166, -0.00728791,\n",
       "          0.06082357, -0.00537404,  0.03332528, -0.06509633, -0.02089403,\n",
       "          0.03292706,  0.00101877,  0.0510809 ,  0.05477249,  0.07478041,\n",
       "         -0.07276022,  0.04048353,  0.02984696,  0.07280662,  0.05768592,\n",
       "         -0.06897536,  0.06543516, -0.1016188 , -0.05013454,  0.01436057,\n",
       "          0.07996077,  0.06045338, -0.08943249, -0.08289234, -0.09445272,\n",
       "         -0.0917815 ,  0.07906624, -0.04912715, -0.08746597,  0.08837538,\n",
       "         -0.10242262, -0.09432936, -0.05098156,  0.05318984, -0.09342098,\n",
       "         -0.08811094,  0.09496946, -0.01205547,  0.09005416, -0.05170618,\n",
       "          0.07965425, -0.06355926,  0.08946715,  0.0701841 ,  0.01106911,\n",
       "         -0.07921416, -0.00586198, -0.07093139, -0.05571615, -0.09570685,\n",
       "         -0.08294483,  0.10156449,  0.08810141,  0.08847762, -0.06849085,\n",
       "          0.03808322, -0.01823393, -0.10235628,  0.06735925, -0.06306846,\n",
       "         -0.03236223, -0.06601474,  0.10650701, -0.02932984,  0.02610057,\n",
       "          0.04268952,  0.04439486, -0.05608631, -0.0875877 , -0.06744913,\n",
       "         -0.05025153, -0.0480884 , -0.07655914, -0.10400078, -0.06631774,\n",
       "          0.02537716,  0.07074804, -0.01678792, -0.01499025, -0.05356828,\n",
       "          0.09865931, -0.05484718,  0.10293259, -0.10538626, -0.05994979,\n",
       "          0.02506403, -0.01381069, -0.08486752, -0.10147281, -0.01415724,\n",
       "         -0.02167669, -0.04138378, -0.07811237, -0.07568555,  0.07546439,\n",
       "         -0.05962276, -0.04591753,  0.01819203,  0.00681051, -0.01061451,\n",
       "         -0.08964039,  0.08384997,  0.01193129,  0.06511291, -0.02141655,\n",
       "         -0.0003387 ,  0.03765   ,  0.0874308 , -0.10177526, -0.07798827,\n",
       "         -0.02491616,  0.10034036,  0.08938778,  0.06986221,  0.06333802,\n",
       "         -0.06758939,  0.10673716,  0.0918544 ,  0.06704409, -0.01049387,\n",
       "         -0.10397964,  0.04459433, -0.05674865,  0.02924498,  0.02555291,\n",
       "         -0.06059154, -0.00760663,  0.0402626 ,  0.09210562,  0.07061929,\n",
       "          0.0992587 ,  0.08551187,  0.09056448,  0.08518671, -0.09439074,\n",
       "          0.01228299,  0.09062467, -0.10565276, -0.09406985, -0.06489315,\n",
       "          0.07310469,  0.03162942, -0.07747364, -0.08266216,  0.00277716,\n",
       "         -0.03755129, -0.06851196,  0.02998076,  0.09461697,  0.03275622,\n",
       "          0.07892226, -0.07969623,  0.06969681,  0.09330302, -0.06589209,\n",
       "          0.00910942, -0.07240221,  0.10662419, -0.08152798, -0.06662944,\n",
       "          0.07763412, -0.05464057, -0.02925145, -0.02522255,  0.01660595,\n",
       "          0.08745205, -0.01829568,  0.06058558, -0.01991504,  0.00570779,\n",
       "          0.01628909,  0.00764092,  0.04065373, -0.00569823,  0.08973347,\n",
       "          0.0285034 , -0.04271245, -0.04339414,  0.09416925,  0.00518313,\n",
       "         -0.0363724 ,  0.00411741, -0.00846383,  0.07830431, -0.08171546,\n",
       "          0.05926593,  0.08176272,  0.00410343,  0.09251819,  0.02461503,\n",
       "         -0.02237175, -0.10116363,  0.04179373, -0.00733022,  0.0464569 ,\n",
       "         -0.09407389,  0.04368985,  0.10700446, -0.09918555,  0.08689972,\n",
       "          0.02030491,  0.05011252, -0.00402433, -0.03748415,  0.1069641 ,\n",
       "         -0.04103616, -0.10338902, -0.00083322,  0.03994679,  0.08614623,\n",
       "         -0.01307566,  0.0018721 , -0.04416473,  0.08122361,  0.09742811,\n",
       "          0.09433862,  0.01141836, -0.0272932 ,  0.10502187, -0.03831887,\n",
       "          0.04020848,  0.05083712]], dtype=float32),\n",
       " array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0.], dtype=float32),\n",
       " array([[-0.02361376, -0.01793896,  0.0598089 , ...,  0.02898362,\n",
       "         -0.06134063,  0.02662736],\n",
       "        [ 0.04458262, -0.03576385,  0.04613996, ..., -0.06607395,\n",
       "          0.02681992, -0.05971859],\n",
       "        [ 0.03929731, -0.01389168, -0.07421191, ...,  0.03646137,\n",
       "          0.06387124, -0.02475978],\n",
       "        ...,\n",
       "        [ 0.05952408,  0.01377954,  0.07068738, ...,  0.02155883,\n",
       "          0.07392914, -0.00705747],\n",
       "        [ 0.05341739, -0.06137065, -0.04243825, ..., -0.0253502 ,\n",
       "          0.05896921, -0.00790192],\n",
       "        [-0.03374526,  0.00091012, -0.00397599, ..., -0.02914729,\n",
       "          0.04182674,  0.06354391]], dtype=float32),\n",
       " array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0.], dtype=float32),\n",
       " array([[ 0.045134  , -0.05750184,  0.06183813, ...,  0.01412484,\n",
       "         -0.01696645,  0.04785117],\n",
       "        [-0.09492163, -0.09237883,  0.00862175, ..., -0.00869731,\n",
       "         -0.06570572, -0.01339182],\n",
       "        [ 0.07717016,  0.00348731,  0.05016131, ..., -0.00269813,\n",
       "         -0.00069132, -0.03376189],\n",
       "        ...,\n",
       "        [ 0.02308481, -0.09029731, -0.05403488, ...,  0.08100846,\n",
       "          0.04356035,  0.01588035],\n",
       "        [ 0.0251194 , -0.10669275, -0.02866279, ...,  0.00330891,\n",
       "          0.03718507, -0.00084209],\n",
       "        [-0.09579649,  0.04074547, -0.10089003, ...,  0.03467111,\n",
       "          0.05641127,  0.04845048]], dtype=float32),\n",
       " array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32),\n",
       " array([[ 0.26860753,  0.23759374,  0.18716115,  0.10115981, -0.02800921,\n",
       "          0.15836516, -0.09860045, -0.13612743, -0.2591631 , -0.06745526,\n",
       "          0.12794834,  0.28275278,  0.26048622,  0.16037929,  0.06893089,\n",
       "          0.27326944, -0.14338836,  0.13564643, -0.09211379,  0.13008103,\n",
       "         -0.26472297,  0.21795341, -0.170733  , -0.27383605, -0.11110888,\n",
       "          0.27600697,  0.22702798, -0.00459588, -0.17530356, -0.05083553,\n",
       "         -0.16051047, -0.19183597, -0.18365046, -0.25116614,  0.14473501,\n",
       "         -0.27543545, -0.17177856,  0.12511957, -0.10618624, -0.14711289,\n",
       "          0.11695391,  0.08973101, -0.08874147, -0.14105387, -0.06558254,\n",
       "          0.19363952,  0.18749887,  0.21695319,  0.11770695, -0.25485963,\n",
       "          0.25919977,  0.14265549,  0.21655783, -0.268605  , -0.19410856,\n",
       "         -0.2635487 ,  0.07162073, -0.23005094,  0.22774038,  0.14635485,\n",
       "         -0.02174345,  0.26865557, -0.28106847,  0.22870532],\n",
       "        [ 0.11814347, -0.27150768,  0.22970602, -0.05107519,  0.13024715,\n",
       "         -0.22511812, -0.07270478,  0.14197227,  0.214019  ,  0.06514168,\n",
       "         -0.20363882,  0.09241399,  0.00876665, -0.26721457,  0.02838585,\n",
       "         -0.2573707 , -0.15197307,  0.07606441,  0.01082382,  0.22011206,\n",
       "          0.03449884, -0.10732807, -0.03040832, -0.11905454,  0.12289959,\n",
       "          0.11914721,  0.11740619,  0.2175462 ,  0.21731564,  0.10062301,\n",
       "         -0.19939868,  0.0307807 ,  0.206593  ,  0.17547199,  0.0771004 ,\n",
       "          0.12980098, -0.14028692, -0.05125089, -0.10067534, -0.17990217,\n",
       "         -0.09905775,  0.02813378,  0.12208158, -0.06715091,  0.158263  ,\n",
       "          0.2726238 ,  0.18609846,  0.0819197 ,  0.11695766,  0.0897949 ,\n",
       "          0.05010843, -0.01414537, -0.09810418, -0.03536871,  0.16851127,\n",
       "          0.19790348,  0.04523122, -0.03120303,  0.20247257,  0.1898247 ,\n",
       "          0.21717307, -0.1722947 , -0.11416198, -0.15105373],\n",
       "        [ 0.10870188,  0.18146837,  0.14324519, -0.08998622, -0.1417022 ,\n",
       "          0.1611493 ,  0.10757065,  0.07891065,  0.26227775,  0.27971438,\n",
       "          0.19699392,  0.17469698, -0.2628698 ,  0.0871329 ,  0.15845254,\n",
       "          0.00669116,  0.17543608, -0.21506211, -0.15887903,  0.10916787,\n",
       "          0.03500929, -0.25696442,  0.10230806, -0.01933074, -0.13693959,\n",
       "          0.01006123, -0.03832127,  0.20600015,  0.06597093,  0.05237749,\n",
       "          0.14187363,  0.20194751, -0.21929261,  0.17882416,  0.19346884,\n",
       "         -0.00202566,  0.22605857, -0.08549339, -0.03458628,  0.16691187,\n",
       "          0.20267859, -0.26618862,  0.03209612,  0.2452828 , -0.13064335,\n",
       "         -0.02258921, -0.2810606 , -0.20288947, -0.18946318, -0.11032897,\n",
       "          0.1573177 , -0.23331484,  0.23096946,  0.15766054,  0.27312967,\n",
       "         -0.24905865, -0.21298832,  0.10883296, -0.08822443,  0.1933406 ,\n",
       "         -0.21681087, -0.06761357, -0.20323184,  0.10348859],\n",
       "        [ 0.14699265, -0.15578681,  0.21832547, -0.2635407 , -0.13177948,\n",
       "         -0.06384574,  0.12653917, -0.02545154, -0.26563758,  0.05637148,\n",
       "         -0.02677238, -0.2614907 , -0.24393818,  0.01459411,  0.27020326,\n",
       "         -0.26167136,  0.2601463 ,  0.08696455, -0.19709799,  0.06995717,\n",
       "         -0.20003894, -0.18288222,  0.27813712,  0.03623322, -0.24461056,\n",
       "          0.03128457,  0.11404881,  0.16709626, -0.13499829,  0.1989502 ,\n",
       "          0.15423596, -0.01219967, -0.19133312,  0.05091348, -0.16285467,\n",
       "          0.09883991, -0.12951334, -0.02203944,  0.04527605,  0.23724416,\n",
       "          0.2426162 ,  0.09582889, -0.01419532,  0.15408206,  0.026072  ,\n",
       "          0.1599125 , -0.22997639, -0.10122497,  0.2621005 , -0.06833653,\n",
       "         -0.18056564,  0.08647519, -0.10307407,  0.16254407, -0.24515252,\n",
       "         -0.07860631, -0.15781908,  0.08387199,  0.20517102, -0.21060099,\n",
       "         -0.2761631 ,  0.21876791, -0.16571294,  0.14103913],\n",
       "        [-0.19156271,  0.15007883,  0.18040895,  0.167938  ,  0.1797427 ,\n",
       "         -0.05551085,  0.06842273, -0.1554471 ,  0.07537037,  0.11489451,\n",
       "          0.0749312 ,  0.19470605, -0.0161669 ,  0.27780083, -0.2672448 ,\n",
       "         -0.09088628,  0.17499629, -0.15415646,  0.07310152, -0.25677925,\n",
       "          0.14243954,  0.22320935, -0.06827664,  0.2656239 ,  0.06171599,\n",
       "          0.10947993,  0.04575452, -0.16232786, -0.10130726,  0.2358031 ,\n",
       "          0.07206213, -0.03115091, -0.16738275,  0.24569008, -0.10968918,\n",
       "         -0.0946946 ,  0.09577042, -0.16894975, -0.2358152 ,  0.00664037,\n",
       "         -0.1858675 ,  0.15693596,  0.17102838, -0.1774809 , -0.10130468,\n",
       "          0.18568522, -0.20586355, -0.19017778, -0.04682316,  0.04715785,\n",
       "          0.05920532, -0.13669477, -0.15515442,  0.22005793,  0.11543661,\n",
       "         -0.04470237,  0.04554108,  0.28389934,  0.16757882,  0.18914846,\n",
       "         -0.1398106 , -0.1492235 ,  0.14906642, -0.07519121],\n",
       "        [-0.23343772, -0.15863028, -0.0699937 ,  0.08598307, -0.06653862,\n",
       "          0.14834136, -0.28469166, -0.22627732, -0.01020122,  0.20110393,\n",
       "          0.19505996, -0.17972083,  0.19578826,  0.27026454,  0.02341902,\n",
       "         -0.18427904, -0.13050269, -0.27919617, -0.04433537, -0.07280669,\n",
       "         -0.13418764,  0.0791977 ,  0.0171251 ,  0.21846089,  0.24317911,\n",
       "          0.10190773, -0.03352192, -0.08512236,  0.05202827, -0.19288847,\n",
       "          0.0637449 ,  0.15580866,  0.05625811, -0.04181844,  0.1717529 ,\n",
       "          0.27321556, -0.03047138, -0.25816786, -0.09062362,  0.15095752,\n",
       "         -0.11467028,  0.156872  ,  0.24428669, -0.24025819,  0.2749004 ,\n",
       "          0.04138532,  0.11629546,  0.13161933,  0.12265438, -0.15621676,\n",
       "          0.13583979, -0.27361012, -0.22210994,  0.08507723,  0.16459665,\n",
       "          0.1462133 ,  0.07977489, -0.06800611,  0.22093353,  0.22819367,\n",
       "         -0.12511806,  0.17428854, -0.21574813,  0.02058837],\n",
       "        [-0.11521521,  0.22161499, -0.19978979,  0.2709634 ,  0.13468269,\n",
       "         -0.03008687,  0.09496534, -0.0848164 ,  0.04161334, -0.13093507,\n",
       "          0.24821511,  0.27224037,  0.16192529, -0.0974405 ,  0.24361077,\n",
       "          0.15258536, -0.13261884, -0.11334337, -0.18132377, -0.21963845,\n",
       "          0.17601776, -0.16244781, -0.13462022,  0.25195608, -0.12975983,\n",
       "         -0.15885268, -0.23188537, -0.21785507,  0.10960099, -0.13147615,\n",
       "          0.20076373,  0.04691774, -0.10891396, -0.2569826 ,  0.07014596,\n",
       "          0.24800524,  0.06692627, -0.16090272,  0.21933678,  0.15850258,\n",
       "         -0.18186782,  0.0853475 ,  0.2331262 , -0.26316863, -0.06909661,\n",
       "         -0.20069903,  0.08887386, -0.20181689, -0.07353465,  0.20761162,\n",
       "          0.25457183, -0.04350685,  0.1384174 ,  0.20333502,  0.12908027,\n",
       "         -0.15223981,  0.18228772, -0.2628357 ,  0.24925765, -0.18500586,\n",
       "          0.01903012, -0.05173595,  0.1259239 , -0.25514558],\n",
       "        [ 0.0548448 ,  0.2734919 ,  0.0033271 ,  0.11728448,  0.17856985,\n",
       "          0.07580644, -0.14564833, -0.07708091, -0.17778558, -0.25914446,\n",
       "         -0.25912592,  0.2019135 ,  0.2602158 ,  0.01191807,  0.07873008,\n",
       "         -0.10360068,  0.16423622,  0.04917252,  0.08695102, -0.2799946 ,\n",
       "          0.24477741, -0.144638  , -0.15580262,  0.17083877, -0.05093378,\n",
       "          0.01159626, -0.04018001, -0.12476571, -0.09342378, -0.19943534,\n",
       "         -0.07864942, -0.1547492 , -0.05476822, -0.04553999, -0.14875412,\n",
       "         -0.18924859, -0.07609822,  0.20164275, -0.21585472,  0.0674881 ,\n",
       "          0.0287821 ,  0.07664752, -0.2629931 ,  0.27803662,  0.20090398,\n",
       "         -0.07905179, -0.21451126, -0.09142153, -0.19636893, -0.1204181 ,\n",
       "         -0.05131836, -0.0223884 , -0.02985489, -0.23500921, -0.1271956 ,\n",
       "         -0.05068387, -0.25048608,  0.08851758, -0.1947884 , -0.14239216,\n",
       "          0.23093233, -0.17388836,  0.22576997,  0.16832185],\n",
       "        [ 0.05888081,  0.07021609,  0.14970633, -0.03766133,  0.2398803 ,\n",
       "         -0.1034981 ,  0.17894989, -0.19355465, -0.17393002, -0.20563531,\n",
       "          0.00406846, -0.01149571, -0.20557292, -0.24165033, -0.20325302,\n",
       "          0.09547979,  0.04864916,  0.23764512,  0.0866541 , -0.11079685,\n",
       "         -0.25767764,  0.08657527, -0.18783656,  0.2758321 ,  0.0657526 ,\n",
       "          0.14903003, -0.05841291, -0.25080022, -0.20048341,  0.2729058 ,\n",
       "          0.17334428,  0.06527454, -0.22577548,  0.27601132, -0.15306894,\n",
       "         -0.21706587, -0.17223239,  0.0185194 , -0.02071342, -0.06245203,\n",
       "          0.23368725,  0.07687792,  0.15665117, -0.2079789 , -0.06018773,\n",
       "          0.18721953,  0.11569893, -0.2357691 ,  0.18748909,  0.27330735,\n",
       "          0.2708815 , -0.07400547, -0.25151107, -0.07396393, -0.15997115,\n",
       "         -0.23790081,  0.15102053, -0.17593813,  0.15405706,  0.26629552,\n",
       "         -0.25251177, -0.25009042, -0.2343953 ,  0.0137085 ],\n",
       "        [ 0.1207101 , -0.24099778,  0.18109077,  0.0789659 , -0.05345999,\n",
       "         -0.1168603 ,  0.04742187,  0.04340753, -0.13663822, -0.17957996,\n",
       "         -0.01937607,  0.26496384,  0.1907742 ,  0.00188419, -0.11328886,\n",
       "         -0.11714469,  0.16534632, -0.19695333,  0.01752475,  0.10567462,\n",
       "          0.257548  ,  0.05781507,  0.2513223 , -0.08447865,  0.01509261,\n",
       "          0.23403963,  0.28277805, -0.02886495,  0.18541679, -0.04650578,\n",
       "         -0.12079455, -0.122601  , -0.09397618,  0.00826502, -0.1426847 ,\n",
       "         -0.06992017,  0.00078845, -0.06647269, -0.13031055, -0.06305178,\n",
       "         -0.26652664,  0.19526601,  0.0603973 , -0.14778961, -0.13818182,\n",
       "          0.05245957, -0.02000493,  0.1307253 ,  0.04483327, -0.23603705,\n",
       "         -0.0929603 , -0.13640143, -0.12034111,  0.0794307 , -0.12707293,\n",
       "          0.00538665, -0.08444273,  0.03589818, -0.03307298, -0.05166046,\n",
       "         -0.2247835 ,  0.26406965,  0.10207501, -0.10109422]],\n",
       "       dtype=float32),\n",
       " array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32),\n",
       " array([[ 0.1984067 ],\n",
       "        [-0.16174594],\n",
       "        [-0.03820705],\n",
       "        [ 0.00291407],\n",
       "        [-0.00072843],\n",
       "        [-0.0215902 ],\n",
       "        [ 0.11620197],\n",
       "        [-0.03412902],\n",
       "        [ 0.07708865],\n",
       "        [ 0.28880876],\n",
       "        [ 0.0235776 ],\n",
       "        [-0.08961138],\n",
       "        [ 0.21659672],\n",
       "        [-0.0395318 ],\n",
       "        [-0.18457213],\n",
       "        [-0.22070822],\n",
       "        [ 0.12585932],\n",
       "        [ 0.15474346],\n",
       "        [ 0.24407536],\n",
       "        [-0.24026723],\n",
       "        [ 0.23705459],\n",
       "        [-0.06091562],\n",
       "        [-0.12028052],\n",
       "        [ 0.03090355],\n",
       "        [-0.02439475],\n",
       "        [-0.10751492],\n",
       "        [-0.2626355 ],\n",
       "        [ 0.08121017],\n",
       "        [-0.06422627],\n",
       "        [-0.06321722],\n",
       "        [ 0.2698133 ],\n",
       "        [-0.05613907],\n",
       "        [ 0.27110595],\n",
       "        [ 0.2305153 ],\n",
       "        [-0.04011062],\n",
       "        [ 0.04619011],\n",
       "        [-0.1101598 ],\n",
       "        [ 0.23752111],\n",
       "        [-0.13009077],\n",
       "        [-0.11471738],\n",
       "        [ 0.07892144],\n",
       "        [-0.196223  ],\n",
       "        [ 0.15970764],\n",
       "        [-0.0335519 ],\n",
       "        [-0.13585399],\n",
       "        [-0.24127454],\n",
       "        [ 0.01772651],\n",
       "        [-0.00642854],\n",
       "        [ 0.24787676],\n",
       "        [-0.28021806],\n",
       "        [-0.08085877],\n",
       "        [-0.16937938],\n",
       "        [-0.20882598],\n",
       "        [ 0.21268088],\n",
       "        [-0.00846988],\n",
       "        [-0.14298314],\n",
       "        [-0.10216017],\n",
       "        [-0.18392852],\n",
       "        [-0.05821112],\n",
       "        [ 0.27884066],\n",
       "        [ 0.12846738],\n",
       "        [-0.20597619],\n",
       "        [ 0.27737337],\n",
       "        [-0.05266732]], dtype=float32),\n",
       " array([0.], dtype=float32)]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.get_weights()   #列出模型的所有权重张量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 编译模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**多层感知器：二进制分类**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**多层感知器：多级分类**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**多层感知器：回归**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**递归神经网络**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "model3.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**在数据上拟合**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/10\n",
      "25000/25000 [==============================] - 68s 3ms/step - loss: 0.4607 - accuracy: 0.7810 - val_loss: 0.4126 - val_accuracy: 0.8170\n",
      "Epoch 2/10\n",
      "25000/25000 [==============================] - 70s 3ms/step - loss: 0.3046 - accuracy: 0.8758 - val_loss: 0.3830 - val_accuracy: 0.8305\n",
      "Epoch 3/10\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.2259 - accuracy: 0.9119 - val_loss: 0.4277 - val_accuracy: 0.8345\n",
      "Epoch 4/10\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.1631 - accuracy: 0.9390 - val_loss: 0.4877 - val_accuracy: 0.8236\n",
      "Epoch 5/10\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.1176 - accuracy: 0.9570 - val_loss: 0.5312 - val_accuracy: 0.8157\n",
      "Epoch 6/10\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.0835 - accuracy: 0.9711 - val_loss: 0.6636 - val_accuracy: 0.8189\n",
      "Epoch 7/10\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.0637 - accuracy: 0.9779 - val_loss: 0.7216 - val_accuracy: 0.8235\n",
      "Epoch 8/10\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.0464 - accuracy: 0.9845 - val_loss: 0.7862 - val_accuracy: 0.8198\n",
      "Epoch 9/10\n",
      "25000/25000 [==============================] - 68s 3ms/step - loss: 0.0368 - accuracy: 0.9882 - val_loss: 0.7752 - val_accuracy: 0.8128\n",
      "Epoch 10/10\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.0271 - accuracy: 0.9913 - val_loss: 0.8713 - val_accuracy: 0.8157\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x1767b4990>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model3.fit(x_train4, y_train4, batch_size=32, epochs=10, verbose=1, validation_data=(x_test4,y_test4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评估模型性能"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**在测试集评估**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000/25000 [==============================] - 11s 429us/step\n"
     ]
    }
   ],
   "source": [
    "score = model3.evaluate(x_test4, y_test4, batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**预测标签与概率**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.3571772e-01],\n",
       "       [9.9999660e-01],\n",
       "       [5.4599410e-01],\n",
       "       ...,\n",
       "       [7.0035458e-04],\n",
       "       [1.5597075e-02],\n",
       "       [9.9907303e-01]], dtype=float32)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model3.predict(x_test4, batch_size=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0],\n",
       "       [1],\n",
       "       [1],\n",
       "       ...,\n",
       "       [0],\n",
       "       [0],\n",
       "       [1]], dtype=int32)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model3.predict_classes(x_test4,batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 保存/加载模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**存储与加载模型**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import load_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "model3.save('model_file.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_model = load_model('model_file.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型微调"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 参数优化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.optimizers import RMSprop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "opt = RMSprop(lr=0.0001, decay=1e-6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "model2.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 早停法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/15\n",
      "25000/25000 [==============================] - 69s 3ms/step - loss: 0.0223 - accuracy: 0.9934 - val_loss: 0.8676 - val_accuracy: 0.8119\n",
      "Epoch 2/15\n",
      "14432/25000 [================>.............] - ETA: 25s - loss: 0.0167 - accuracy: 0.9951"
     ]
    }
   ],
   "source": [
    "from keras.callbacks import EarlyStopping\n",
    "\n",
    "early_stopping_monitor = EarlyStopping(patience=2)  #最多等待2轮，如果效果不提升，就停止\n",
    "\n",
    "model3.fit(x_train4, y_train4, batch_size=32,\n",
    "            epochs=15,\n",
    "            validation_data=(x_test4,y_test4),\n",
    "            callbacks=[early_stopping_monitor])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
